The Area Activation Model of Saccadic Selectivity in Visual Search

We present an approach towards a simple, explicit model of saccadic selectivity in visual search tasks. The model in its present state includes weights for target-distractor similarities and fixation field size as its only adjustable parameters. Based on these, the model predicts the statistical distribution of saccadic endpoints for any given visual search display. Besides providing an explicit and complete mathematical specification of the model, we demonstrate the performance of its computer simulation in a triple-conjunctive search task. The model successfully simulates empirical data reported by Williams and Reingold (in press). Modeling Visual Search How do we detect a prespecified target item among a set of distractors? Numerous studies employing the paradigm of visual search have attempted to answer this question (see Treisman, 1988 and Wolfe, 1998, for reviews). In a typical visual search task, subjects have to decide whether a search display contains a designated target item, indicating their decision by pressing either a “yes” or a “no” button. In most studies, reaction times (RTs) and error rates were analyzed as a function of the number of items in the display (display size). The majority of current models of visual search were based on data obtained within this paradigm. An early attempt to model visual search is the Feature Integration Theory (Treisman & Gelade, 1980; Treisman, 1988). This theory proposes the existence of preattentive feature maps, one for each stimulus dimension such as color or shape. These maps are created in parallel after stimulus onset and allow immediate target detection if the target is defined by a unique feature in any single dimension (feature search). If the target is defined by a specific combination of features (conjunctive search), attention is necessary to locally combine the information of the corresponding feature maps. As a result, subjects have to inspect the display in an item-by-item fashion until target detection or exhaustive search. The Feature Integration Theory thus explains the finding that reaction time tends to increase with display size in conjunctive search tasks, while it is almost constant in feature search tasks. A more recent approach is the Guided Search Model (Cave & Wolfe, 1990; Wolfe, Cave & Franzel, 1989; Wolfe, 1994), which proposes a two-stage model of visual search. In the first, parallel stage, an activation map containing likely target locations is created on the basis of both topdown and bottom-up sources of activation. The second, serial stage uses the activation map to guide visual attention from item to item, starting with the item with the highest activation, then proceeding to the second highest, and so on, until the target is found or the current activation falls below a certain threshold (see Chun & Wolfe, 1996). Besides many variations of these two models, there are also more complex approaches like the one by Grossberg, Mingolla and Ross (1994). Their model uses artificial neural networks to achieve perceptual grouping of search displays into subregions. Visual search is assumed to proceed serially between these subregions and in parallel within them. Recently, several researchers have analyzed participants’ eye movements during visual search to supplement traditional RT and accuracy measures (e.g. Findlay, 1997; Hooge & Erkelens, 1999; Jacobs, 1987; Luria & Strauss, 1975; Motter & Belky, 1998; Rayner & Fisher, 1987; Scialfa & Joffe, 1998; Shen, Reingold, & Pomplun, in press; Viviani & Swensson, 1982; Williams, Reingold, Moscovitch, & Behrmann, 1997; Williams & Reingold, in press; Zelinsky, 1996; see Rayner, 1998, for a review). Some of these studies have further examined saccadic selectivity, i.e. the proportion of saccades directed to each distractor type, by assigning saccadic endpoints to the closest display item. Such studies have found a strong selectivity towards distractors sharing a particular feature with the target item (e.g. Findlay, 1997; Hooge & Erkelens, 1999; Luria & Strauss, 1975; Motter & Belky, 1998; Scialfa & Joffe, 1998; Shen, Reingold & Pomplun, in press; Williams & Reingold, in press; but see Zelinsky, 1996). Given that eye movements are usually accompanied by shifts of attention (see Hoffman, 1998, for a review), it seems that subjects can selectively attend to a critical subset of items in the display rather than perform an item-by-item search as suggested by the original Feature Integration Theory. To date, no explicit model has been proposed which allows for simulating saccadic selectivity in visual search. In the present article, we propose such an approach, referred to as the Area Activation Model. Following the description of the model, we examine its performance by simulating the saccadic selectivity findings reported by Williams and Reingold (in press). The Area Activation Model The Area Activation Model is based on assumptions concerning three aspects of visual search performance: (1) the extent of available resources for processing, (2) the choice of fixation positions, and (3) the scan-path structure. Processing resources -The extent of available resources for processing is determined by a two-dimensional Gaussian function with its peak centered at the current gaze position (e.g. Pomplun, Ritter & Velichkovsky, 1996). The standard deviation σf of the Gaussian function would be affected by a variety of factors such as task difficulty, item density, and item heterogeneity, but in essence should be a function of the area from which information is extracted during a fixation (henceforth “fixation field”). For example, if the target and distractors are easily discriminable and the density and heterogeneity of items are low, we would expect the fixation field to be larger than when discriminability is low and density and heterogeneity are high. This theoretical measure is likely to be correlated with the number or density of fixations in a given area. If the fixation field is smaller, we would expect more fixations per display area. In fact, in the current simulation we are using the empirically observed number of fixations per trial to adjust σf. Fixation positions Fixation positions are chosen to optimize the amount of information acquired. However, the execution of saccades entails a certain amount of error, which causes fixations to deviate from these optimal positions. Another source of error in empirical data is related to inaccurate measurement of eye movements. It is important for a valid comparison between empirical and simulated data to consider both saccadic error and measurement error. For every point in the display it is possible to calculate its informativeness or relevance to the search task, creating an activation map. In the present simulation, we use weights corresponding to features along several dimensions to determine activation for individual items. A variety of models may suggest different activation maps (e.g. Cave & Wolfe, 1990; Wolfe, 1994). In order to make the method transparent and applicable to a wide variety of tasks, we provide a general, explicit specification of the model. A search display consists of N items with positions (xn, yn) and features fn along D dimensions, n∈{1,…, N}, d∈{1,…, D}. The search target has the features t. Each dimension d is assigned a weight w, which currently has to be estimated on the basis of the results from a pilot-study. If, for example, subjects rely entirely on color, the color weight should be set to 1 and all other weights set to 0. If an item n is identical to the target in dimension d, the item's feature activation an is set to the weight of that dimension: } , , 1 { , } , , 1 { , otherwise , 0 if , ) ( ) ( ) ( ) ( D d N n t f w a d d n d d n ∈ ∈ = = The total activation of item n is then calculated as the sum of its feature activations, implying the possibility of simultaneous guidance of attention by two or more dimensions: